What Are The Factors Influencing Criminality? Evidence from US

---

Overview and Motivation

When thinking at welfare and government expenditure in health, we often focus on physical health and not much in mental health. Starting this project out main idea was to study the relationship between Mental Health Services and Homicides. Later, we moved to a broader view to study which are the factor that mostly impact criminality in a developed country.

After looking up for relevant data-sets on the internet, we decided to concentrate on United States of America, looking for data for each state.

The main motivation behind our project is our interest in social sciences and policies. Indeed, before starting, we decided for this topic because, possibly, our results will be interesting for a policy maker in taking decisions on education and mental health services expenditure and provision, as well as other factors.

Related Work

Project Objectives

The objective of this project is to understand whether the variables we have found have an impact on criminality and if they do, how they affect it (negatively or positively and significantly or not).

We will focus on education, mental health expenditure by the governments, GDP and composition of the population.

Our aim is to identify potential policy targets in dealing with criminality and, consequently, quality of life in a Country/State.

Project’s Outline

The project has been organized in different phases, all carried through the software R. In this section we will explain how we organized our work for you to have a clear idea of our project.

We start by searching relevant data-sets on the internet and looking for variables of interest. Then, we proceed by cleaning and tidying each of them. This process is done by trying to have each data-set in the same form, to ease a later analysis of them. We create the whole final data-set with all the wanted information from the different sources.

Then, we begin the Explanatory Data Analysis exploiting visualization tools and summarizing main characteristics of the data-set.

Finally, we analyze the results of panel data regressions and previous findings to answer our research questions and sum up the main take-home messages.

Research questions

The research questions we will try to answering throughout our project are:

• Is there any relationship between expenditure for mental health by the government and criminality?
• Is the level of education and wealth (through GDP) of a State relevant for its level of criminality?
• Is the composition of the population, in terms of both age and ethnicity, relevant for criminality in the area?
• Is mental health expenditure affected by how much the population is educated or by GDP of the country?

Given the questions posed above, the answers we will search for in our project could lead a reader to question himself on how to exploit these presences of correlation to reach a lower level of criminality. Though, the latter consideration makes sense only if we are able to find significant relationship between the different variables.

In this part we present the data we use to analyse and answer our research questions. We start by importing and cleaning them. We use a total of 17 files to form our final data-set, we present them below separately, following the themes. Notice that we clean each data-set such that they all appear “standardized” (for example we select years from 2004 to 2013, since only in this time-framework we have all data available). This is done to facilitate the join process among all data-set.

Crime Data

Sources and Description

We used the dataset on estimated crimes (from 1979 to 2019) available in the FBI website. We have repeated observations for each state in the United States of America from 1979 to 2019.

The data-set contains 2116 observations for 15 variables, which are

• year the year of the observation
• state_abbr, state_name the abbrevation and the name of the State. Notice that the first line of each year is blank. These observations refer to the total, i.e. United States.
• population the number of population in a given year and State.
• violent_crime, homicide , rape_legacy, rape_revised, robbery, aggravated_assault, property_crime, burglary, larceny, motor_vehicle_theft, violent_crime, caveats each refers to the respective number of registered crimes by the FBI or of caveats. From the source we learn that homicides resulting from the events of September 11, 2001, are not included. This is fine for us, because it will be an outlier not significant for our analysis.

Missing values for each feature

	Missing Values
year	0
state_abbr	41
state_name	41
population	0
violent_crime	0
homicide	0
rape_legacy	156
rape_revised	2116
robbery	0
aggravated_assault	0
property_crime	0
burglary	0
larceny	0
motor_vehicle_theft	0
caveats	2045

From above we can see how many NAs we have for each feature. Looking at this we already decide to not take into account rape_revised and caveats, while we already now that state_name and state_abbr missing values refers to United States, so we will fill them appropriately.

Wrangling/cleaning

To clean this dataset we have to transform year values into numeric. Moreover, we change the name of column state_name to State and selected only some crime which we think could be more relevant for our study and could be more impacted by mental health expenditure. We also replaced NAs in State and state_abbr with “United States” and “US”.

The cleaned dataset is called “estimated_crimes” and is reported below:

Mental Health Expenditure

Sources and Description

For this part we have to download data-set for each year separately from 2004 to 2013 and you can find them at this link.

Since the structure for each year’s data-set is the same we report only the first one, for year 2004:

The data-set for each year contains 51 observations for 3 variables, which are

• Location the State or US
• SMHA Expenditures Per Capita is the State Mental Health Agency data on expenditures in mental health per capita in each state
• population the number of population in a given year and State.
• footnotes which are notes on the data such as the fact that the reporting period reflects spending in state fiscal year, which may vary by state. Data are not adjusted for inflation and Puerto Rico is included in the US’s total.

Wrangling/cleaning

We cleaned data-set for each year and then we joined them. To cleaned them we remove the dollar sign $ from the expenditure per capita values, as well as transform them into numeric. We also change its name of mental health expenditure per capita to the respective year of the dataset, i.e. 2004. This is done to ease the join process, which is made by State, which is the renamed previous Location.

The resulting dataset on mental health is “mh_exp” and is reported below:

We can also look at how many NAs are present:

There are some, and if you look at data you can see that the missing value comes usually from Puerto Rico’s observation.

US demographic

Sources and Description

We used the 3 dataset in the United States Census Bureau’s website.

The first one is about race composition from 2000 to 2010

This one contains 364 observations for 18 variables, which are

• REGION,DIVISION, STATE, NAME which identifies the region, division, state code and name of the state.
• RACE is the race, it goes from 0 to 6, with 0 being the total and 1-6 identifying ethinicities as White, Black/African-American and so on
• POPBASE2000 and POPESTIMATEyear for each year, which are the estimated population in a State in a given year, for the respective race cathegory

The second one is about age and sex composition from 2000 to 2010

This one contains 13572 observations for 19 variables, which are

• REGION,DIVISION, STATE, NAME which identifies the region, division, state code and name of the state.
• SEX is the sex, it can be either 0 (total), 1 (male) or 2 (female)
• AGE is the age, it goes from 0 to 85 years old, then there’s also 999 which is the total population
• POPBASE2000 and POPESTIMATEyear for each year, which are the estimated population in a State in a given year, for the respective sex and age cathegory

The third one is about race, age and sex composition from 2010 to 2019

This one contains 236844 observations for 21 variables, which are

• SUMLEV is the identification of the summary levels used by the census, it is also called “area type”
• REGION,DIVISION, STATE, NAME which identifies the region, division, state code and name of the state.
• RACE is the race, it goes from 1 to 6, 1-6 identifying ethinicities as White, Black/African-American and so on
• ORIGIN is the origin, it can be 0 (total), 1 (Not Hispanic) or 2(Hispanic), although this sata is absent between 2000 and 2009 so we will omit it
• SEX is the sex, it can be either 0 (total), 1 (male) or 2 (female)
• AGE is the age, it goes from 0 to 84 years old, then we have 85 which comprises 85+ years old
• POPBASE2010 and POPESTIMATEyear for each year, which are the estimated population in a State in a given year, for the respective race, sex and age cathegory

Wrangling/cleaning

The cleaning is done for each data-set separately. Later we proceed to join them. In all data-set we trasform REGION into a factor and we rename the levels such that total US replaces 0, North-East (NE) replaces 1, Mid-West (MW) replaces 2, South (S) replaces 3, and West (W) replaces 4.

Also RACE and SEX will become factors with respective levels labels: (White=1, BlackAfricanAmerican=2, AmericanIndianAlaska=3, Asian=4, HawaiianPacificIslanders=5, Racegreaterthan1=6 and Total= for the dataset in years 2000-2010) and (Total=0, Male=1, Female=2).

Names of variables are also changed slightly to have them in line with other data-sets and through pivot_longer and pivot_wider we adjust the structure of the table in a standardized way.

Moreover, for AGE we created some sub-groups instead of having the complete range 0-85 years old. The age groups we create are 0-17, 18-24, 25-44, 45-64, 65-84 and 85+. We still don’t know whether age composition has an impact on criminality, but we consider important to have the subgroups 18-24 and 25-44, since in education, as we will see, these two age groups are considered.

Also race groups are different in the cleaned dataset: White, BlackAfricanAmerican, Asian and Other_race. The latter comprises all the other cathegories. We also filter for years of interest (2004-2013).

In 2011-2013 we miss the observation for United States, which instead is present between 2004-2010. Therefore, we created a dataset for it by taking the sum across states, since US’s values would be the total and we put everything together for years 2011-2013 to obtain.

We end up having two data-sets on demographics, one for the years 2004-2010 and the other from 2011 to 2013. Finally, we join these obtaining the final “demographics” dataset:

No missing value is present. Although, notice that in the cleaning process we have to fill some Region’s values which otherwise would be missing. But, knowing the data-set and the State, it is straightforward.

Education

Source and Descriptions

For education we decided to look up for a proxy: Bachelor’s degree incidence in the population. We found two data-sets, one for the percentage of people between 25-44 years old with a Bachelor’s Degree, for years 2005-2018, and one for the number of bachelor’s conferred in the age range 18-24 per 1000 individuals, for years 2000-2018.

The former is:

This one contains 53 observations for 15 variables, which are

• State which identifies the state, or the whole US
• 2005 … 2018 one column for each year observed

This one contains 53 observations for 20 variables, which are

• State which identifies the state, or the whole US
• 2000 … 2018 one column for each year observed

Wrangling/Cleaning

Both data-sets are cleaned separately and then put together. The main task in both is to create a new variable year and another, respectively perc_bscholder_25_44 and perc_bscconferred_18_24, therefore resulting in longer data-sets.

Notice, that the datas we had in the second data-set referred to 1000 people and was not in percentage terms as instead is perc_bscholder_25_44, therefore, to obtain perc_bscconferred_18_24 we have to divide by 1000 and multiply by 100 the data.

Another thing which is worth mentioning is the fact that in the data-set describing %25-44 years old people with a Bachelor’s Degree, we miss observations for 2004. To adjust for it we, first, create these observations as NAs, then fill them with the value from 2005. In our opinion this shouldn’t alter our analysis, because the difference from year to year is relatively small.

Then, we join the two cleaned data-sets in “edu”:

There are no missing values in this data-set, although remember that the ones we had in edu_percholder_25_44 have been filled with 2005’s values.

GDP

Source and Descriptions

The data-set on GDP can be found in the Bureau of Economic Analysis, of U.S. Department of Commerce, website.

This has 483 observations for 27 columns, which are:

• GeoFips and GeoName which identify the state, or the whole US through a code and the name, respectively
• LineCode and Description which identify the kind of variable we are looking at, i.e. Current-dollar GDP (millions of current dollars), Real GDP (millions of chained 2012 dollars), etc… They are of 8 different kind, but we will focus on Current-dollar GDP as you will see
• 1997 … 2019 one column for each year observed

Wrangling/Cleaning

In order to clean the dataset we filter for one values of Description only, since it’s the one of interest for us: Current-dollar GDP (millions of current dollars). We delete the column which are not relevat, remaining with renamed GeoName, which is now State, and 1997 … 2019. We use “pivot_longer” to create two variables year and Current_dollar_GDP_millions increasing the length of the data-set. Of course, we also filter for years in the frame 2004-2013.

The resulting dataset is “GDP_Cleaned”:

No missing values are present in “GDP_Cleaned”.

Final Dataset

Now that we have each data-set cleaned and wrangled in a “standardized” way we can join them by State and year.

Although, simply joining them produces NAs and by looking at the data we understand that this happens because some data-set considered also Divisions and this makes appear among the States also New England, Mideast, Great Lakes, Plains, Southeast, Southwest, Rocky Mountain and Far West. We filter them out, as well as Puerto Rico. Indeed, the latter presents many missing values too.

The resulting dataset is called “project”:

Data Overview

In this section we are going to do an explanatory data analysis by using the cleaned data described in the data part. Throughout the section, we will still need some transformation of the data to facilitate the visualization and to understand everything in a deeper way.

To present the crimes and the race in a nicer way, we decide to mutate the former in term of “per 1000 inhabitants” and the latter in percentage terms. This makes sense also because different countries have different dimensions and population size. Therefore, maintaining absolute magnitudes would probably give us a wrong perception and result. We don’t change the variables’ names, though.

As you may have seen in the section on data, we end up having many features. Although some of them might be irrelevant or redundant. To see that we use a straightforward correlation command; this can be already a step towards the selection of most important variables that we may need for our analysis later.

The main findings are:

• All variables describing the population, such as population, Female and Male, as well as age, are perfectly (or at least highly) correlated. For this reason, we can select population and ignore the amount of population which is female or male. Also because these values are always around 50% of population in each state, it wouldn’t be too informative. Notice that, White and Black African-American seem negatively correlated, as well as Age_0_17 and Age_over85. These are only two examples, but the motivation is straightforward, i.e.: if the population is very young, it can’t be old at the same time.
• White and crimes have a negative corellation, except for rapes, although for this the correlation seems small.
• Black/African-American is positively related with all crime, while Asian has only low correlations with them.
• Mental Health expenditure per capita appears positively correlated with education of the population, while its correlation with GDP doesn’t seem relevant. Its correlation with crimes is dubious, we will better investigate on it with some visualization tools.
• GDP tends to be positively correlated with crimes, with exception of rapes. We will deepen this result later.
• It seems that a young population (18-44) leads to higher homicides, aggravated assaults and violent crimes. Meanwhile, older population (45+) appears negatively related with crimes.
• As you know, we have considered two proxies for Education until now, although they are highly correlated and it doesn’t make sense to use both. Therefore, we decide to use perc_bscholder_25_44.
• Let’s consider also correlations among crimes. As we would expect, the correlation between the different crimes is positive, indicating that there’s little differentation. So, whenever criminality in a state is high, the level of all crimes is, more or less, high. Although, among them, rape seems to be the less correlated with the others.

Moving forward, having observed the correlations above, we can also look into each variable.

To do so we can look up easily at the outcome of the data-set’s summary.

For age:

Age’s variables summary statistics

	Min.	1st Qu.	Median	Mean	3rd Qu.	Max.
Age_0_17	0.1676	0.2288	0.2401	0.2394	0.2495	0.3150
Age_18_24	0.0829	0.0967	0.0999	0.1010	0.1032	0.1446
Age_25_44	0.2308	0.2542	0.2643	0.2662	0.2761	0.3680
Age_45_64	0.1144	0.2519	0.2623	0.2609	0.2714	0.3122
Age_65_84	0.0589	0.1075	0.1151	0.1140	0.1214	0.1609
Age_over85	0.0050	0.0151	0.0174	0.0177	0.0205	0.0269

The highest percentage of population is between 25 and 64 years old while the lowest has more than 85 years.

For race:

Race’s variables summary statistics

	Min.	1st Qu.	Median	Mean	3rd Qu.	Max.
White	0.2557	0.7373	0.8339	0.8020	0.8908	0.9654
BlackAfricanAmerican	0.0041	0.0326	0.0770	0.1145	0.1566	0.5812
Asian	0.0060	0.0138	0.0229	0.0372	0.0407	0.4083
Other_race	0.0122	0.0210	0.0273	0.0463	0.0443	0.3396

The majority of the population is white, followed by black and African-American.

For crimes:

Crimes’ variables summary statistics

	Min.	1st Qu.	Median	Mean	3rd Qu.	Max.
homicide	0.0084	0.0269	0.0454	0.0494	0.0611	0.3487
violent_crime	0.8655	2.6797	3.5792	4.0563	5.0244	15.3711
rape_legacy	0.0972	0.2576	0.3131	0.3271	0.3830	0.8914
aggravated_assault	0.5119	1.5712	2.2706	2.5683	3.3108	8.0413

Remember that crimes are expressed in per 1000 terms.
Homicides are the less common crime, while violent crimes and aggravated assault occur on average to 4 and 2.5 people out of 1000.

For mental health expenditure, education, population and GDP:

Other variables summary statistics

	Min.	1st Qu.	Median	Mean	3rd Qu.	Max.
mh_exp_pc	2.423e+01	7.144e+01	9.883e+01	1.201e+02	1.451e+02	4.099e+02
perc_bscconferred_18_24	1.939e+00	4.586e+00	5.505e+00	5.661e+00	6.397e+00	1.374e+01
perc_bscholder_25_44	1.948e+01	2.572e+01	2.987e+01	3.056e+01	3.408e+01	6.535e+01
Current_dollar_GDP_millions	2.266e+04	7.300e+04	1.738e+05	5.605e+05	3.818e+05	1.678e+07
population	5.091e+05	1.715e+06	4.352e+06	1.173e+07	7.092e+06	3.160e+08

In this last summary table, it’s worth mentioning that

• The variablility of mental health expenditure per capita seems high.
• Population and GDP are not really interesting withouth further analysis and grouping by state or region, since the size of states can be very different, impacting these two variables.

Univariate visualizations

To present the most important data by State we created an interactive map which shows the selected variable distribution in US’s states in a given year.

Moreover, we try to analyze graphically the main variables separately in order to potentially detect outliers or interesting path/characteristics.

We start with a time series for mental health expenditure per capita, both for the whole US and the single regions. To do so we compute the median value in each region for every year and create a time series on R. Then we plot the whole thing in one graph:

Mental Health Expenditure (per capita)

We can see that in general, the expenditure per capita has increased from 2004 to 2013, with some ups and downs throughout the period. The downward sloping part are especially relevant in two regions, West and South between 2009 and 2010/11. We don’t have enough data, but a possible explanation could be the financial crisis which had impact on government budgeting. The largest difference between 2004 and 2013 values is observed for North-East, while the smallest is for South, of which gap between these years is of $7 circa. In the time series we only look at the median. It could be interesting to observe the same data through a boxplot to understand variability and outliers.

We start by looking at each regions and US in total.

We notice that US has a low variability, but here data for US are already considered as a total, it doesn’t consider each state observed. Instead, for the regions we capture, as before, that North-East is the one with largest variation, and we already know from the time series that this is due to the steadily increase in mh_exp per capita over the years.

The boxplots are ordered by median and we can see how North-East is the one with greatest median and how US’s median (which we can consider as the mean median across regions) is second for magnitude. Thus, it’s driven significantly by North East states expenditure.

South and Mid-West are the regions in which states seem to spend less for mental health expenditure in per capita terms.

We can clearly observe some outliers. But you can notice that they are quite clustered. Probably each group of outliers represents a state’s obervations in different years. These are not a problem for our analysis, therefore we just continue.

The second boxplot we propose is to shed the light on each region’s state.

As we expected, in regions such as South and West, where we observed outliers in the boxplot before, there are states which appear far from the others. These are District of Columbia and Alaska. The latter is indeed on the west coast, but it’s somehow detached from the other states of the region. Also District of Columbia is a case on its own since it’s not a proper state but a federal district.

We confirm that Mid-West is the region with less variability among its states in mental health expenditure per capita.

Demographic: Age and Race Composition of the Population

Let’s continue our univariate visualization part with demographics variables.

We do so by exploiting barplots. Again, we group results by regions as it can give us an idea of the distribution of population among the different US’s areas. Of course, we continue to look also at the total US. To group results by region we took median values and computed percentages of the population.

We start with a barplot for race composition of the population:

We immediately observe that between total US and North-East the difference is minimal. Although, no large difference is present for any of the region. In all of them there is a high prelevance of white people. The percentage for them is the highest in Mid-West area, while there’s a particular high percentage of Black/African-American population in the South.
Moreover, while the group “other race” is a minority everywhere, it is not in the West, where instead Black/African-American percentage is lower than both asian and other races.

Now on age composition:

The same results on overall observations throughout US as we had on the summary table in the data overview section return here. What’s new is the fact that we can make consideration on the “age” of each region. Although the composition of the population does not change in a relevant way.

Demographic: Education

Again, we group results by region and we took median values of the percentage of bachelor’s degree holder with age 25-44. We can notice from the following graph that, using our proxy for education, we have a lower percentage of bachelor’s holder in the South. Instead, North-East seem has 6% more educated people than the mean value of US.

We also look at a boxplot to understand the variability of education inside each region. The variability is not too high, although we observe some outliers in South, again we think they are due to District of Columbia:

Criminality Distribution Across States

Again, we group results by region we took median values and transform values in per 1000 terms. So, finally we ask ourselves the distribution of crimes in US.

South and West have the highest level of criminality, with a great departure from other regions for violent crimes and aggravated assaults. Violent crimes seem to be the most common crime, while homicide is the least frequent and it is the lowest in Mid-West.

Multivariate visualizations

Now that we discussed variables by themselves, we can start to see the various relationships that exist between multiple variables at the same time. Notice that when appropriate we use a log10 scale. This is useful for some of our variables because they cover a large range of values. We also decide to remove District of Columbia and United States, since in most cases the first creates outliners and is not a proper state and because US are just a total observation.

Since from the corrplot in the first part of the EDA section the correlation between mental health expenditure and criminality appeared dubious we start investigating this relationship through a scatterplot. We consider mental health expenditure per capita against the various kinds of criminality: homicides, violent crime, rape and aggravated assault.

From the scatterplot above we can see that the overall correlation is slightly negative. Which means that for an increase in public mental health per capita spending there is, on average, a decrease in criminality.

Remember that from the corrplot we have identified a positive correlation between education and mental health expenditure per capita. The higher the education the higher is the spending for health. We tried to show this through a scatterplot and the outcome is exaclty what we expected by the corrplot, even if we don’t rule out the District of Columbia.

This second scatterplot that we propose is criminality against education.

Here we can see two distinct things. First of all the correlation is negative, thus, on average, the higher the education the lower is the criminality rate. The second thing we can notice is about the log GDP. In all the criminalities, except rape, the lighter dots (higher GDP) lies above the tendency line, while the darker dots (lower GDP) lies below. Therefore, there is a positive correlation between GDP and the kind of crimes we considered, except for rape, that has a negative correlation.

Now we want to see a few of the correlation we saw before, but in the time dimension.

First of all the effect of mental health expenditure on criminality over time. We decided to report here the time series for only one crime, homicide, since the patterns are similar for all the four of them:

From this time series we can see how in the US the number of total homicides decreases over time. This can possibly follows from an increase in the mental health.

Now we check the mental health spending against the education over time.

Here we see that the increase in education over the selected decade also correspond to an increase in the public expenditure in mental health.

Finally we check the homicides against the education. Again, the pattern is similar also for other type of crimes, therefore we report only the one for homicides:

This final time series shows that in the decade of interest the decrease of homicides also correspond to an increase in education.

However to better understand all of these effect and draw stronger conclusions we should do some panel data analysis on the data-set as we will do in the next section.

Up to now we could only try to guess why such a correlation exist, and which are the social factor that induce such a result.

Such opinions for the correlations are the following:

• Right now he have found three effects that we want to discuss for one last time. There is a negative correlation between mental health spending and violent crimes, we think that this is a reasonable correlation since taking care of possible dangerous people could reduce the impact on criminality, or at least reduce the relapse.
• There is a negative correlation between education and criminality, this also can be reasonable as a correlation since the education not only increases the hard skills, but also teaches people how to live in a civilized society, as well as it creates networks (easier to get help) and awareness on social problems.
• Finally there is a positive correlation between GPD and crimes, except for rape. At a first glance we thought this wasn’t a good instance because a wealthier state will have less criminality than a poorer state. However after thinking a little bit about the possible social reasons behind it we thought that this might be caused by the social distance between poor and rich people. In a wealthier state the distance between wealthier and poorer might be high. This might induce more people to commit violent crimes to gain money or reduce debt. This might also explain why there is a positive effect for rape, in fact out of the four effect that we considered this is the one that is least related to possible wealth change in the individual that commits the crime. Although, these are only supposition, therefore we looked up and research finding a vast literature on how higher GDP has on average a positive effect on criminality. It results that there could be some simultaneous causality considerations to do, since GDP, education, unemployment and poverty are strictly linkes factors.
  Some references are Effect of GDP on Violent Crime, Northrup, Klaer, The Relationship between Crime and Economic Growth in Malaysia: Re-Examine Using Bound Test Approach, 2016, Mulok, Kogid, Lily, Asid

• Answers to the research questions
• Different methods considered
• Competing approaches
• Justifications

• Take home message
• Limitations
• Future work?